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sions  of  the  simplest  and  fundamental  problems  of  at¬ 
tributed  decision  making  and  j>e  that,  apart  frem 

a  few  exceptions,  such  problems  are  hard  (HP-complete, 
or  worse) .  Sooe  of  the  problmu  studied  are  the  «tU- 
fcxkown  team  decision  problem,  the  distributed  hypothesis 
testing  problem,  as  well  as  the  problem  of  designing 
a  coenunlcatlans  protocol  that  guarantees  the  attain¬ 
ment  of  a  prmmpmUfirnd  goal  with  as  little  communica¬ 
tions  as  possible.  These  results  indicete  the  inherent 
difficulty  of  distributed  decision  making,  even  for 
very  simple  problems,  with  trivial  centralised  counter¬ 
parts  and  suggest  that  optimality  may  be  an  elusive 
goal  of  distributed  systems.  - - - - 

1.  introduction  and  Motivation 

If*  this  paper  we  formulate  and  study  certain  simple 
decentralised  problems.  Our  goal  is  to  formulate  pro¬ 
blems  which  reflect  the  inherent  difficulties  of  decen¬ 
tralisation;  that  is,  any  difficulty  in  this' class  of 
problems  is  distinct  from  the  difficulty  of  correspond¬ 
ing  centralised  problems.  This  is  accomplished  by 
formulating  decentralised  problems  whose  centralised 
counterparts  are  either  trivial  or  vaclous. 

One  of  our  goals  is  to  determine  a  boundary  between 
"easy”  and  "hard*  decentralised  problems.  Our  results 
will  indicate  that  the  set  of  "easy*  problems  is 
relatively  small. 

Xll  problems  to  be  studied  are  imbedded  in  a  dis¬ 
crete  framework;  the  criteria  we  use  for  deciding 
whether  a  problem  is  difficult  or  not  come  frexn  com¬ 
plexity  theory  (Carey  and  Johnson,  1979;  Papsdimitriou 
and  Steiglitz,  1992)  *  following  the  tradition  of  com¬ 
plexity  theory,  problems  that  may  be  solved  by  a  poly¬ 
nomial  algorithm  are  considered  easy;  NP -complete,  or 
worse,  problems  are  considered  hard.*  However,  an 
HP— completeness  result  does  not  close  a  subject,  bit  la 
rather  as  a  result  which  can  guide  research;  further 
research  should  focus  on  special  cases  of  the  problem 
or  on  approximate  versions  of  the  original  problem. 

The  maim  issue  of  interest  in  decentralised  system 
may  be  loosely  phrased  as  "who  should  communicate  to 
whom,  what,  how  often  etc.*  Tras.  a  purely  logical 
point  of  view,  the  first  question  that  has  to  be  raised 
is  *are  there  any  complication  necessary?*  Any 
further  questions  deserve  to  be  studied  only  if  we 
comm  to  the  conclusion  that  communications  are  Indeed 
necessary. 

The  subject  of  Section  2  is  to  characterise  the  in¬ 
herent  difficulty  of  the  problem  of  deciding  whether 
any  coastal  cation*  are  necessary,  for  a  given  situa¬ 
tion.  He  adopt  the  following  approach;  a  decentral¬ 
ised  system  exists  in  order  to  accomplish  a  cartels 
goal  which  is  externally  specified  end  well-known.  A 


set  of  processors  obtain  (possibly  conflicting)  obser¬ 
vations  on  the  state  of  the  environment .  Each  processor 
has  to  make  a  decision,  based  on  his  own  observation. 
However,  for  each  state  of  the  environment,  only  certain 
decisions  accomplish  the  desired  goal.  The  question 
*a re  there  any  communications  necessary?*  may  be  then 
reformulated  as  "can  the  goal  be  accomplished,  with 
certainty,  without  any  commications?"  K#  show  that 
this  problem  is,  in  general,  a  hard  one. 

we  than  impose  some  more  structure  on  the  problem, 
by  assuming  that  the  observations  of  different  proces¬ 
sors  are  related  in  a  particular  way.  The  main  Issue 
that  we  address  is  "bow  much  structure  is  required  so 
that  the  problem  is  an  easy  ona?"  and  we  try  to  deter¬ 
mine  the  boundary  between  easy  and  hard  problems. 

In  Section  3  we  formulate  a  few  problems  which  are 
related  to  the  besic  problem  of  Section  2  and  discuss 
their  complexity. 

In  Section  4  we  study  a  particular  (more  structured) 
decentralised  problem  -  the  problem  of  decentralised 
hypothesis  testing  -  on  which  there  has  been  some  in¬ 
terest  recently,  and  character  ire  its  difficulty. 

Suppose  that  it  has  bean  found  that  communications 
are  necessary.  The  next  question  of  interest  is  "whet 
is  the  least  amount  of  conmunl cations  needed?”  This 
problem  (Section  5)  is  essentially  the  problem  of  desig¬ 
ning  an  optimal  cocaunications  protocol;  it  is  again  a 
hard  one  and  we  discuss  some  related  issues. 

In  Section  6  we  present  our  conclusions  and  discuss 
the  conceptual  significance  of  our  results.  These  con¬ 
clusions  may  be  sumarlred  by  saying  that: 

a)  Even  the  simplest  (exact)  problems  of  decentralised 
decision  making  are  hard. 

b)  Allowing  sooe  redundancy  in  communications,  may  * 
greatly  facilitate  the  (off-line)  problem  of  desig¬ 
ning  s  decentralised  system. 

c)  Practical  coosunications  protocols  should  not  be  ex¬ 
pected  to  be  optimal,  es  far  as  minimisation  of  the 
amount  of  coomuni  cations  is  concerned. 

Some  of  the  results  of  this  paper  appear  la 
(Papediaitriou  and  Tsitsiklis. .  1983)  and  (almost)  all 
proofs  may  be  found  in  [Tsitsiklis,  1993] • 

2.  A  Problem  of  Silent  Coordination 

In  this  section  we  formulate  and  study  the  problem 
whether  a  set  of  processors  with  different  Information 
may  accomplish  a  given  goal  -with  certainty-  without 
any  coBsuaicatlon*. 

Let  m)  be  a  set  of  processors.  Each  processor, 

say  processor  i,  obtains  an  Observation  y  which  comes 
from  a  finite  set  of  possible  observations .  Then, 

processor  i  makes  a  decision  u^  which  belongs  to  a 

finite  set  0.  of  possible  decisions,  according  to  a 

nj#.  i 
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following : 
Theorcz  2.1: 


ui  “  <yl*  ' 

where  is  soae  function  from  Y  into  .  The 
«- tuple*  <y1#...,y  )  is  the  total  information  avail¬ 
able;  so  it  &ay  be  viewed  as  the  "state  of  the  envi¬ 
ronment  . "  For  each  state  of  the  environment,  we  assume 
that  only  certain  M-tuples  of  decision  ac¬ 

complish  a  given,  externally  specified,  goal.  More 
precisely,  for  each  (y  ....jy^d  Y^x — xYM  we  are  given 

a  set  S(y1#....yM)  C  O^x.-.xO^  of  satisficing  decisions. 

(So,  S  nay  be  viewed  as  a  function  from 
0 

Y,xY^  x.  ..x  Y  into  2 
12b 

The  problem  to  be  studied,  which  we  call  "distri¬ 
buted  satisficing  problem"  (efter  the  term  introduced 
by  H.  Simon  11980))  may  be  described  formally  as 
follows: 

Distributed  Satisficing  (PS):  Given  finite  sets  Y^,..., 
Ym#  0I#...fDK  end  e  function  $:  Yx  x...x  YM  ♦ 

0  x.  ..xu*, 

2  A  ,  ere  there  functions  3^ :  Y^  ■*  0^ ,  1*1 , 

2,  •  • • fH,  such  that 


t8i ' •  •  •  •  Vyn))cs(yi' •  •  •  *  v,yi' •  •  • » V e 

x...x  ym  (2.2) 


Remarks? 

1.  We  are  assuming  that  the  function  S  is  "easily 
computable ; "  for  example,  it  may  be  given  in  the  form 
of  e  table, 

2.  The  centralised  counterpart  of  DS  would  be  to 

allow  the  decision  u.  of  each  agent  depend  on  the 
entire  set  (yx . y*)  of  observations;  so,  ^  would 

be  e  function  from  Y  x...x  Y  into  0. .  (This  cor- 
1  H  * 

responds  to  a  situation  in  which  all  processors  share 
the  seme  information.).  Clearly,  then,  there  exist 
satisfactory  (satisficing)  functions  ^  :Y^x. .  .xY^ 

if  and  only  if  S( yx».».*y  )**  •  •***• 

Since  $  is  an  "easily  computable"  set  as  a  function 
of  its  arguments,  we  can  see  that  the  centralized 
counterpart  of  DS  is  a  tiivial  problem.  So,  any  dif¬ 
ficulty  inherent  in  DS  is  only  caused  by  the  feet  that 
information  is  decentralized. 

3.  A  "solution*  for  the  problem  DS  cannot  be  a  closed- 
form  formula  which  gives  an  answer  o(no)  or  l(yes). 
Rather,  it  has  to  be  an  algorithm,  a  sequence  of  ins¬ 
tructions,  which  starts  with  the  data  of  the  problem 

,  Uir...,D^#S)  and  eventually  provides  the 

correct  answer,  accordingly,  the  difficulty  of  the 
problem  DS  may  be  characterized  by  determining  the 
piece  held  by  OS  in  the  complexity  hierarchy.  Tor 
definitions  related  to  computational  complexity  and 
the  methods  typically  used,  the  reader  is  referred  to 
(Carey  and  Johnson,  1979;  Papadiaitriou  and  Steiglitz, 
1982) . 

4.  If,  for  seme  i,  the  set  Vi  is  a  singleton,  proces¬ 
sor  i  has  no  choice,  regarding  his  decision  and,  con¬ 
sequently  ,  the  problem  is  equivalent  to  e  problem  in 
which  processor  1  is  absent.  Hence,  without  loss  of 
generality,  ve  only  need  to  study  instances  of  DS  in 
which  joji  2,  yA. 

5.  We  believe  that  the  problem  DS  captures  the  es¬ 
sence  of  coordinated  decision  making  with  decentral¬ 
ized  information  and  without  communications  (silent 
coordination)  • 

Some  initial  results  on  DS  are  given  by  the 


e)  The  problem  DS  with  two  processors  1»2)  and  res¬ 
tricted  to  instances  for  which  the  cardinality  of  the 
decision  sets  is  2  (|u^|»2,  i-1,2)  cay  be  solved  in 

polynomial  time. 

b)  The  problem  DS  with  two  processors  (h*2)  is  HP- 
complete,  even  if  we  restrict  to  instances  for  which 
IOjJ-2.  |0j|-3. 

c)  The  problem  DS  with  three  (or  more)  processors 
(M>3)  is  KP-camplete,  even  if  we  restrict  to  instances 
for  which  |0i(-2,Yi 

Theorem  2.1  states  that  the  problem  DS  is,  in  gen¬ 
eral,  a  hard  combinatorial  problem,  except  for  the 
special  case  in  which  there  are  only  two  processors 
and  each  one  has  to  make  a  binary  decision.  It  should 
be  noted  that  the  difficulty  is  not  caused  by  en  at¬ 
tempt  to  optimize  with  respect  to  e  cost  function, 
because  no  cost  function  has  been  introduced.  In  game 
theoretic  language,  we  are  faced  with  e  "game  of  kind," 
rather  than  a  "game  of  degree." 

We  will  now  consider  some  special  cases  (which  re¬ 
flect  the  structure  of  typical  practical  problems)  and 
examine  their  computational  complexity,  trying  to  deter¬ 
mine  the  dividing  line  between  easy  and  hard  problems. 
From  now  on  we  restrict  our  attention  to  the  case  in 
which  there  axe  only  two  processors.  Clearly,  if  e 
problem  with  two  processors  is  hard,  the  corresponding 
problem  with  three  or  more  processors  cannot  be  easier. 

We  have  formulated  above  the  problem  DS  so  that  all 
pair*  <yl'y2*€  YjXY^  are  likely  to  occur.  So,  the 

information  of  different  processors  is  completely  un¬ 
related;  their  coupling  is  caused  only  by  the  structure 
of  the  satisficing  sets  S(y^,y^).  In  most  practical 

situations,  however,  information  is  not  completely  uns¬ 
tructured:  when  processor  1  observes  y^,  he  is  often 

able  to  make  certain  inferences  about  the  value  of  the 
observation  y  of  the  other  processor  and  exclude  cer¬ 
tain  values.  We  now  formalize  these  ideas: 

Definition:  An  Information  Structure  1  is  a  subset 
of  Y^xY^*  w*  say  that  an  information  structure  I  has 

degree  (D^,E>2  are  positive  integers)  if 

(i)  For  each  y^CY^  there  exist  at  most  D^  distinct 
elements  cf  Y^  such  that  (y^,y  )€  2. 

(ii)  For  each  y^€Y^  there  exist  at  most  D^  distinct 
elements  of  Y^  such  that  (y^y^)?  2. 

liii)  D^.D^  axe  the  smallest  Integers  satisfying  U), 

Ui).  An  information  structure  2  is  called  classical 
if  D^D^-1;  nested  if  or  D2*l. 

We  now  interpret  this  definition:  The  information 
structure  1  is  the  set  of  pairs  (y^,y^>  of  observation 

that  may  occur  together.  If  I  has  degree  (D^  ,D^) 

processor  1  may  use  his  own  observation  to  decide  which 
elements  of  Y^  may  have  been  observed  by  processor  2. 

In  particular,  he  may  exclude  all  elements  except  for 
D  of  them.  The  situation  faced  by  processor  2  is 
symmetrical. 

If  D^*l  and  processor  2  observes  y^,  there  is  only 

one  possible  value  fox  y_.  So,  processor  1  knows  the 
observation  of  processor *2.  (The  converse  is  true 
when  D  *1).  This  is  celled  e  nested  information  struc¬ 
ture  because  the  information  of  one  processor  contains 
the  information  of  the  other. 

When  D  •$  *1,  each  processor  knows  the  observation 
of  the  other;  so,  their  information  is  essentially 


3.  jtUttd  Problems 


. 

ih«r«d. 

Since  pairs  (y.ty.)  not  in  I  cannot  occur,  there  is  no 
meaning  in  requirlng^the  processors  to  make  compatible 
decisions  if  (y^,/^)  were  to  be  observed.  This  leads 

to  the  following  version  of  the  problem  DS: 

DSI:  Given  finite  sets  Y  ,Y  .UwU*,  IC  Y.xY  and  a 
0  xU  1  2  *  12 

function  S:  1^2  1  ,  are  there  functions 

i-1,2,  such  that 

<^1<y1)#S2<y2))€s(yi»y2)*  *(Yi'y2)€IY  .  U.*> 

Note  that  any  instance  of  DS1  is  equivalent  to  an  ins¬ 
tance  of  05  in  which  Sly^y^-  2. 

That  is,  no  compatibility  restrictions  are  placed  on 
the  decisions  of  the  two  processors,  for  those  (y^.y  ) 
that  cannot  occur.  2  2 

We  now  proceed  to  the  main  result  of  this  Section: 

Theorem  3. 2. It 

a)  The  problem  DSI  restricted  to  instances  satisfying 
any  of  the  following: 

(i)  One  or  more  of  |u^|,|u2|#  D^,D2  is  equal  to  1. 

(il>  |0a|-|0a|-2. 

Uii)  Dx-P2-2, 

(iv)  O^jUjl-2,  (or  P^luJ-2) 
may  be  solved  in  polynomial  time. 

b)  The  problem  DSI  is  NP-complete  even  if  we  restrict 
to  instances  for  which 

|ox|-  Dj-3,  |02|-D2-2 

The  result  concerning  the  case  D^«l  or  D^»l  is  not 

surprising.  It  is  well-known  that  nested  information 
structures  msy  be  exploited  to  solve  otherwise  dif¬ 
ficult  decentralised  problems.  But  except  for  the  case 
D^*D^«2  (which  is  sort  of  a  boundary)  the  absence  of 

nestedness  makes  decentralised  problems  computationally 
hard.  Our  result  gives  a  precise  meaning  to  the  state¬ 
ment  that  non -nested  information  structures  are  much 
more  difficult  to  handle  than  nested  ones. 

Theorem  3.2.2  shows  that  even  if  arc  held  cons¬ 

tant,  the  problem  DSI  is,  in  general,  NP-complete . 

There  is,  however,  a  special  case  of  DSI,  with  D^,D2 

constant,  for  which  an  efficient  algorithm  of  the 
dynamic  programing  type  is  possible: 

Theorem  3.2.3:  bat  Y^»{l, .. . ,»),  Y2»{l,...,n)  and  sup- 

POM  that  |i-j|<  0.  YU. jiff  I.  »wa.  if  C  is  h.14 
constant,  DSZ  may  be  solved  in  polynomial  time. 

Mmark:  Zn  fact,  the  conclusion  of  Theorem  3.2.3  re¬ 
mains  true  if  we  assume  mwn  and  we  replace  the  condition 
|i- j |<^  D  by  the  weaker  condition  |i-j |  (mod  n)<^  D. 

The  proof  consists  of  a  small  modification  of  the 
preceding  one. 

The  condition  |i-j |<^  D,  YU#  j)*  2  is  fairly  natural 
in  certain  application?.  For  example ,  suppose  that  the 
observations  y^  and  y^  art  noisy  msasursmsnta  of  an 

unknown  variable  x  (y  where  the  noises  v  axe 

'  i  i  i 

boundsds  |vj<  D/2. 

The  condition  |i-j|  (mod  n><^  D  msy  else  arias  if  the 
observations  y^y^  are  noisy  measurements  of  some 

unknown  angle  t  ♦  w^ 


In  this  Section  we  define  and  discuss  briefly  a  few 
more  combinatorial  problems  relevant  to  decentralized 
decision  making.  All  of  them  will  be  seen  to  be  harder 
than. problem  DS  of  the  last  section  (i.e.  they  contain 
DS  a*  *  special  case)  and  are,  therefore  KP-hard  (that 
is,  NP-ceoplete,  or  worse). 

The  best  known  static  decentralized  problem  is  the 
team  decision  problem  [Maxschak  and  Badner,  1972) 
which  admits  an  elegant  solution  under  linear  quadratic 
assumptions.  Its  discrete  version  is  the  following: 

TDP  (Team  Decision  Problem)  t  Given  finite  sets  Y^,  Y^ 
°l'°2*  *  Prob4l>ility  »ass  function  p:  Y^xY^C,  and  a 
cost  function  c:  Y^Y^^xUj  -N,  find  decision  rules 
*2  **  '  i“l#l  which  minimize  the  expected  cost 


J<dl'V  v?l  e‘yl'y2'^ltyl)'^t5r2,,p‘yl*y2» 
yl  1  *2  2 

s  <*!'*;}  J“*<WeUlXV  e^.y^^.u^-O).  If 
we  solve  TDP,  we  have  effectively  answered  the  question 
whether  there  exist  such  that  This 

is  equivalent  to  the  question  whether  there  exist  sat¬ 
isficing  decision  rules  (with  the  satisficing  sets 
S(Yi#Y2)  defined  as  above).  Therefore,  TDP  is  harder 

than  DS*. 

Proposition  3.1:  The  discrete  team  decision  problem 

is  NP-hard,  even  if  the  range  of  the  cost  function 
c  is  {0,1}. 

Instead  of  trying  to  •’satisfice"  for  every  pair  of 
ob»emt ions  (y^#y2>€  Yi'xY2*  ^  t&ay  be  more  appro¬ 
priate  to  impose  a  probability  mass  function  on  Y^xY^ 

and  try  to  maximize  the  probability  of  satisficing. 

This  leads  to  the  next  problem: 

KPS  (Kaximize  Probability  of  Satisficinc) :  Given 

finite  sets  *2'*2'U1'^2'  *  probability  mass  function 

U  xD 

Ps  Y1*Y2^C  and  a  function  Sr  Y  BY  *-2  1  *,  find 

decision  rules  Y^-n^,  i»l,2,  which  maximize  the 
probability  of  satisficing  J(^,^)  • 

Pr  ( (aa  ( yx  >  #  d2  (y2 ) )  €S  ^ .  y2 ) ) . 

We  now  take  a  slightly  different  point  of  view. 
Suppose  that  communications  are  allowed,  so  that  the 
processors  may  always  make  satisficing  decisions  by 
communicating  (assuming  that  S (y1.,y2>f<^, 

Y(yi»y2>€  YjXYj).  Suppose,  however,  that  communica¬ 
tions  are  very  expensive,  so  that  we  ere  interested 
in  a  scheme  which  guarantees  satisficing  with  a  mini  - 
w»  amount  of  cosnunications .  We  will  assiae  that  if 
one  of  the  processors  initiates  a  communication,  all 
their  information  will  be  exchange  et  unity  cost. 

(For  a  more  refined  way  of  counting  the  amount  of  com¬ 
munications,  see  Section  3.5.) 

MFC  (Minimise  Probability  of  Communications) :  Given 
finite  sets  Y2,T2'U1'Q2  4  probability  mass  function 

0  xo 

pt  y1*T2^°  4  Junction  S:  YX*Y2*2  1  2,  find 

decision  rules  Y^O^Ulc)#  i-1,2,  which  minimize 
the  probability  Prte^y^-c  or  a2(y2>-c)  of  ccemualca- 
ting  subject  to  the  constraint 


If  and  c^(y2)f*cj  then  (y^  »b2  ly^)  )€S  (y1#y2). 

The  proof  of  the  following. is  trivial: 

Preposition  3.2:  The  problems  KPS  and  MFC  are  NP-hard. 
In  fact,  we  also  have: 

Proposition  3.3:  The  problems  TOP  (with  a  zero-one 
cost  function)  and  KPS  are  HP-hard,  even  if  |u^|»|  )**2  . 

We  could  also  define  dynastic  versions  of  DS  or  of 
the  tew  pro b lea,  in  a  straightforward  way  (Tenney, 

1983] .  Since  dynaaic  problems  cannot  be  easier  than 
static  ones,  they  are  automatically  KP-hard.  „ 

4.  Decentralized  Hypothesis  Testing 

A  basic  problem  in  decentralised  signal  processing, 
which  has  attracted  a  fair  amount  of  attention  recently, 
is  the  problem  of  decentralised  hypothesis  testing 
(Tenney  and  Sandell,  1981;  Ekchian,  1982;  Ekchian  and 
Tenney,  1982;  Kushner  and  Facut,  1982;  Lauer  and 
Sandell,  1983}.  A  simple  version  of  the  problem,  in¬ 
volving  only  two  processors  and  two  hypotheses  may  be 
described  as  follows; 

Two  processors  and  receive  observations  y^CY^, 
y^ry^,  respectively,  where  Y^  is  the  eet  of  all  pos¬ 
sible  observations  of  processor  i.  (Figure  1) .  There 

are  two  hypotheses  8  and  B  on  the  state  of  the 
o  l 

environment,  with  prior  probabilities  p^  end  p^  res¬ 
pectively.  For  each  hypothesis  B  ,  we  are  also  given 
the  joint  probability  distribution  Fiv^y^l®^ 

observations,  conditioned  on  the  event  that  is  true. 
Upon  receipt  of  y^,  processor  evaluates  e  message 

u^GlO.l)  according  to  the  rule  where 

d^Y^O,!}.  Then,  u ^  and  u^  axe  transmitted  to  a 


figure  1:  A  Scheme  for  Decentralized  Hypothesis 
Test inf. 


central  processor  (fusion  center)  which  evaluates 
uo-uxO  u^  end  declares  hypothesis  to  be  true  if 

u  *0,  H,  if  u  *1.  (so,  we  essentially  have  a  voting 
o  1  o 

scheme).  The  problem  is  to  select  the  functions 

$2  so  es  to  minimize  the  probability  of  accepting  the 
wrong  hypothesis.  (More  general  performance  criteria 
may  be  also  considered) . 

Most  available  results  as  state  that 

p(y1*y2|Hi)-p<y1|8i)f«y2|K1).  *-x.a.  «-u 

which  states  that  the  observations  of  the  two  proces¬ 
sors  axe  independent,  when  conditioned  on  either  hy¬ 
pothesis.*  Xn  particular,  it  has  been  shown  (Tenney 
and  Sandell,  1981}  that  the  optimal  decision  rules  ^ 

ere  given  in  terms  of  thresholds  for  the  livelihood 
PC’<H  |y.) 

ratios  - -  r  r-  .  The  optimal  thresholds  for 

'**2 1  » 

the  two  sensors  are  coupled  through  a  system  of  equa¬ 
tions  which  gives  necessary  conditions  of  optimality. 
(These  equations  are  precisely  the  person-by-person 
optimality  conditions).  Few  analytical  results  are 
available  when  the  conditional  independence  assosptioo 
is  removed  [Lauer  and  Sandell,  1983]  .  The  approach  of 
this  section  is  aimed  at  explaining  this  status  of  af¬ 
fairs,  by  focusing  on  discrete  (and  finite)  versions 
of  the  problem. 

We  first  have: 

Theorem  4.1:  Zf  Y^, Y?  are  finite  sets  and  (4.1)  holds, 

then  optimal  choices  for  £  **Y  be  found  in  poly¬ 

nomial  time. 

So,  under  the  conditional  independence  assumption, 
decentralised  hypothesis  testing  is  a  computationally 
easy  problem.  Unfortunately ,  this  is  not  the  case  «toea 
the  independence  assumption  is  relaxed.  Our  main  re¬ 
sult  (Theorem  4.2)  states  that  (with  Y^,  finite 

sets) ,  decentralized  hypothesis  testing  is  a  hard  com¬ 
binatorial  problem  (NF-hard) .  This  is  true  even  if  we 
restrict  to  the  special  case  where  perfect  detection 
(zero  probability  of  error)  is  possible  for  the  corres¬ 
ponding  centralized  hypothesis  testing  problem. 

Although  this  is  in  some  sense  a  negative  result,  it  is 
useful  because  it  indicates  the  direction  in  which 
future  research  on  this  subject  should  proceed: 

Instead  of  trying  to  find  efficient  exact  algorithms, 
research  should  focus  on  approximate  algorithms,  or 
exact  algorithms  for  problems  with  more  structure  than 
that  assumed  here.  Moreover,  our  result  implies  that 
any  necessary  conditions  for  optimality  to  be  developed 
axe  liXely  to  be  deficient  in  one  of  two  respects: 

a)  Either  there  will  be  a  very  large  number  of  deci¬ 
sion  rules  satisfying  these  conditions. 

b)  Or,  it  will  be  hard  to  find  decision  rules  satis¬ 
fying  these  conditions. 

Xn  particular,  optimal  decision  rules  are  not  given  in 
terms  of  thresholds  on  livelihood  ratios. 

Of  course,  there  remains  the  question  whether  ef¬ 
ficient  approximate  algorithms  exist  for  the  general 
decentralized  hypothesis  testing  problem,  or  whether 
we  must  again  Vestxict  to  special  cases  of  the  problem, 
we  now  present  formally  the  problem  to  be  analyzed. 

PHT;  (Decentralized  Hypothesis  Testing,  Restricted  to 
Instances  for  which  Perfect  Centralized  Detection  is 
Possible) .  ^ 

We  are  given  finite  sets  Y^Y^;  a  rational  amber 

number  k;  a  rational  probability  mass  function 
p:  Y^Y^oniO.l),  *  partition 

*  Such  an  assumption  Is  reasonable  in  problems  of  detac- 
tion  of  a  known  signal  in  independent  noise,  but  is  typ- 
signal  **  P* oblems  of  detection  of  an  unknown 


.U^)  o f  YjXYj.*  Do  there  exist  JjiT^O#!), 
d2sY2-{0.l)  such  that  '?<d1«d2’lx*  wh#r* 

Ji&.b-)  -  £  ny1.y2)81(y1)S2<y2)  ♦ 

1  2  (Vy2)C*o 

«^j,€Xi  ’‘W^WWl7  (<-2> 

Bmrks:  1.  Zf  we  lot  k-0,  then  DHT  is  *  special  cam 
of  problem  DS  (Section  2),  with  |t>1|*|u2]»2.  And  is 

polynoeial ly  solvable,  According  to  Theorem  3. '2.1.  In 
general  DHT  is  a  special  caac  of  KPS  end  TOP  (Section 
3.3)  with  |DaHu2h2.  Consequently,  Theorem  4.2 

below  proves  Proposition  3.3. 

2)  Clearly*  the  optimisation  problem  (Minimise  J(^^) ♦ 
with  r Aspect  to  <^,d2)  cennot  be  AAsier  then  DHT. 

Since  DHT  will  be  shown  to  be  HP-complete,  it  follows 
thet  the  Above  opr  Ini  ration  problem  is  HP-hard. 

3)  In  DHT,  as  defined  Above#  we  exe  only  considering 
instenees  for  which  perfect  centralized  detection  is 

possible:  Think  of  B  as  being  the  hypothesis  thet 
o 

O',  ,yj€  A  ,  end  JL  as  being  the  hypothesis  thet 
12  0  i. 

(Yl'y2)€  *1*  c«rtAiniy'  **  *  processor  knows  both  y^ 
y2 ,  the  true  hypothesis  may  be  found  with  certeinty. 

Tor  the  decentralised  problem,  the  cost  function 

is  eesily  seen  to  be  the  probebility  of  error. 

4)  The  result  to  be  obteined  below  remains  velid  if  the 
fusion  center  uses  different  rules  for  combining  the 
messages  it  receives  (e.g.  uo*(u^v(  ^)))«  i*  ** 

leave  the  combining  rule  unspecified  end  try  to  find 
an  optimal  combining  rule. 

Theorem  4.2:  WIT  is  HP-complete. 

5.  On  Design Communications  Protocols 

Suppose  that  we  Are  given  An  instance  of  the  dis¬ 
tributed  satisficing  problem  (DS)  and  that  it  was 
concluded  that  unless  the  processors  ccemunicate, 
satisficing  cannot  be  guaranteed  for  all  possible  ob¬ 
servations.  Assuming  that  communications  axe  allowed 
(but  are  costly) ,  we  have  to  consider  the  probles  of 
designing  a  communications  protocol:  what  should  each 
processor  cosmunicete  to  the  other*  and  at  whet  order? 
Moreover*  since  communications  are  costly*  we  are 
interested  in  e  protocol  which  minimises  the  total 
maber  of  binary  messages  (bits)  that  have  to  be  com¬ 
municated.  (The  word  "bits*  above  does  not  have  the 
information  theoretic  meaning.) 

Before  proceeding#  we  must  make  more  precise  the 
notion  of  a  communication  protocol  and  of  the  masher 
of  bits  than  guarantee  satisficing. 

Given  en  instance  tk*(yi#y2*Di#02*X,S)  of  the  problem 

DSI  we  will  say  that: 

There  is  a  protocol  which  guarantees  satisficing 
with  0  bits  of  coeauni  cations »  if  V  is  a  YZS  instance 
of  the  problem  DSI.  (Thet  is*  if  there  exist  satis¬ 
ficing  decision  rules*  involving  no  cq— nai cations.) 
we  then  proceed  inductively: 

There  is  s  protocol  which  guarantees  satisficing 
with  X  bits  of  coeauni  cations  (XS  V)  *  if  for. soma 
if{l*2)  (say#  i-1)  there  is  e  function  miY  ~{0*l}* 

•ueh  that  for  each  of  the  instances.  * 

p*-(Yinm"i(0).Y2#D1*02#X  OCCYjOn  (0))*Y2)#S)  and 

•  That  TTTjJ  VV*2  B  ’ 


p--<*1n»-1<i),Y2,uJ.uJ.j  th«r*  u 

e  protocol  which  guarantees  satisficing  with  not  more 
than  K-l  bits  of  coeauni  cations.  (Here  »"1(i)» 

The  envisaged  sequence  of  events  behind  this  defini¬ 
tion  is  the  following:  Each  processor  observes  his 
measurement  y^HY, ,  i«l*2.  Then,  one  of  the  processors* 

say  processor  1*  transmits  a  message  m(y^)*  with  a 
single  bit  to  the  other  processor.  Fromthst  point  on* 
it  has  become  com  ion  knowledge  that  y^ey^Hm*!  (y^)  i 

tharefore*  the  remaining  elements  of  Y  may  be  ignored. 
We  can  now  state  formally  the  problem  of  interest: 

MBS  (Minimum  bits  to  satisfice):  Given  an  instance  V 

of  D6I  end  X£  H,  is  there  e  protocol  which  gurantees 
satisficing  with  not  more  than  X  bits  of  rwum  1  cations? 

By  definition*  MBS  with  K-0  is  identical  to  the 
problem  DSI.  Moreover*  MBS  with  X  arbitrary  cannot  be 
easier  than  MBS  with  X-0  (which  is  a  special  case) . 
Therefore*  MBS  is,  in  general  HP-hard.  Differently 
said*  problems  involving  mmnuni  cations  are  at  least 
as  hard  as  problems  involving  no  communications. 

We  have  seen  in  Section  2  that  when  1 0^  |  •  1 0^  |  *2 , 

D5Z  may  be  solved  in  polynomial  tima.  Therefore,  MBS 
with  X-0,  |UjJ«2*  |o2 1  “2  is  polynomially  solvable. 

However,  for  arbitrary  X*  this  is  no  longer  true: 

Theorem  5.1:  KBS  is  NP-complete ,  even  if  |o2|-|02|* 

{0,1}  and  even  if  we  restrict  to  instances  for  which, 
for  any  (y^y^CZ,  either  S(y1#y2)«  {(0,0)}  or 

S(y1#y2)-{(1*1)}. 

The  above  theorem  proves  e  conjecture  of  A.  Yeo 
[Yao*  1979] .  The  proof  was  mainly  constructed  by 
C.  Papadiaitrlou  end  may  be  found  in  (Papadimitriou 
end  Tsitslklis*  1962). 

We.  should  point  out  thet  the  special  case  referred 
to  in  Theorem  5.1  concerns  the  problem  of  distributed 
function  evaluation:  we  axe  given  e  Boolean  function 
f:Y  xY  -*{o,l}  end  we  require  thet  both  agents  (proces¬ 
sors  eventually  detaxmine  the  value  of  the  function 
(given  the  observation  -input  ly^,y2)),  by  exchanging 

e  minimum  number  of  bits.  In  our  formalism; 

S,yl'y2)“  1<0'0,J  if  f,yl'y2,“°  S(yl'y2)"* 

if  f(y1.y2)-i. 

In  Section  2  we  had  investigated  the  complexity  of 
DSI  by  restricting  to  instances  for  which  the  set  I 
had  constant  degree  (D^,D2).  This  may  be  done*  in 

principle,  for  MBS,  as  well*  but  no  results  are  avail¬ 
able*  except  for  the  simple  case  in  which  D^-D2*2. 

In  feet*  when  D.-D-2  oach  processor  may  transmit 
his  information  to^the  other  agent  by  coeauni eating  a 
single  binary  message  end,  for  this  reason*  we  have: 

Proposition  5.1:  KBS  restricted  to  instances  for  which 
D-D  -2  may  be  solved  in  polynomial  time.  Moreover# 
an  optimal  protpcol  requires  transmission  of  at  most 
two  binary  messages*  one  from  each  processor. 

When  (D^,D2)  is  larger  than  (2*2),  there  is  not 

much  we  can  say  about  optimal  protocols.  However#  it 
is  easy  to  verify  that  there  exist  fairly  simple  non- 
optimal  protocols  (which  may  be  calculated  in  poly¬ 
nomial  time)  which  involve  relatively  small  amounts  of 
coeauni  cation .  This  is  because: 

Proposition  5.2:  Suppose  that  2  has  degree  (D^iO^)  end 
that  S(y1*y2>^,  Y(y1*y2>CI.  Then  information  may  be 

centralised  (end  therefore  satisficing  is  guaranteed) 
by  means  of  a  protocol  requiring  coeauni  cation  of  et 
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*>«•  |i«>Sj(nx  DjJj  binary  m»U9*>  by  each  processor. 

Moreover,  such  s  protocol  nay  be  constructed  in  time 
0((|Yl|-|Y2l)(|Y1l*jY2|)).  (Hurt  Jxj  .  *€  R.  tuadi 

for  the  nellest  Integer  larger  then  x.) 

Remark  1;  It  night  be  teapting  to  guess  that  proces¬ 
sor  1  (respectively  2)  needs  to  cosnunicate  only 
|iog2D2J  (respectively  bits,  but  this  is  not 

true,  as  can  be  seen  iron  fairly  simple  examples. 

6.  Conclusions 


suaaarire  here  the  main  conclusions  of  this  pa- 

P«- 

Even  if  *  set  of  processors  have  complete  knowledge 
of  the  structure  of  a  distributed  decision  making  pro¬ 
blem  and  the  desired  goal;  evan  if  the  corresponding 
centralised  problem  is  trivial;  evan  if  all  relevant 
sets  are  finite,  a  satisficing  decision  rule  that  in¬ 
volves  no  on-line  cc— uni  cations  may  be  very  hard  to 
find,  the  corresponding  problem  being,  in  general, 

NP -complete.  There  arc  many  objections  to  the  idea 
tba  KP -completeness  is  an  unequivocal  measure  of  tha 
difficulty  of  a  problem,  because  it  is  based  on  a 
worst  case  analysis,  whereas  tha  avaraga  performance 
of  an  algorithm  might  ba  a  more  adequate  measure; 
moreover  KP-hard  optimisation  problems  may  have  very 
simple  approximate  algorithms.  However,  HP-complete 
problems  are  of  tan  characterised  by  the  property  that 
any  known  algorithm  is  vary  close  to  systematic  ex¬ 
haustive  search;  they  do  not  possess  any  structure  to 
be  exploited. 

Concerning  the  problem  D S,  and  its  variations,  we 
may  reach  the  following  specific  conclusions:  No  sim¬ 
ple  algorithm  could  solve  DS.  Given  that  ccmunica- 
tions  would  be  certainly  required  for  those  instances 
of  DS  that  possess  no  satisficing  decision  rules,  it 
would  not  be  a  great  loss  if  we  allowed  the  proces¬ 
sors  to  cosnunicate  even  for  some  instances  of  DS  for 
which  this  would  not  be  necessary.  Even  if  these 
extra  coenuni cations  -being  redundant-  do  not  lead  to 
better  decisions,  they  may  greatly  facilitate  the  de¬ 
cision  process  and  -from  a  practical  point  of  view  - 
remove  some  load  from  the  computing  machines  employed. 

Concerning  the  probl^  of  distributed  hypothesis 
testing,  we  have  shown  that  it  becomes  hard,  once  a 
simplifying  assumption  of  conditional  independence  is 
r amoved.  This  explains  why  no  substantial  progress 
on  this  problem  had  followed  the  work  of  Tenney  and 
Sand* 11  (19821. 

From  a  pore  ganeral  perspective ,  we  are  in  a  posi¬ 
tion  to  say  that  the  basic  (and  the  simplest)  problems 
of  decentralised  decision  making  are  hard,  in  a  precise 
mathematical  sense.  Moreover,  their  difficulty  does 
not  only  arise  when  one  is  interested  in  optimality. 
Difficulties  persist  even  if  optimality  is  replaced 
by  satisficing.  As  s  consequence,  further  research 
should  focus  on  special  cases  and  easily  solvable 
problem  as  well  as  on  approximate  versions  of  the 
original  problems. 

In  cases  where  coeounlcations  are  necessary  (but 
costly)  there  arises  naturally  the  problem  of  desig¬ 
ning  s  protocol  of  ccemuni cations .  Unfortunately,  if 
this  problem  is  approached  with  the  intention  to  mi¬ 
nimise  the  amount  of  communications  that  will  guar¬ 
antee  the  aceeopllshment  of  a  given  goal,  we  are  again 
led  to  intractable  combinatorial  problems.  Therefore, 
practical  cosnunieations  protocols  can  only  be  desig¬ 
ned  on  s  "good"  heuristic  or  ad-hoc  basis,* sad  they 
should  not  be  expected  to  be  optimal;  approximate 
optimality  is  probably  s  more  meaningful  goal.  Again, 
allowing  some  redundancy  in  on-line  communications 
may  lead  to  substantial  savings  in  off-line  computa¬ 
tions. 
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